Tooth shade analyzer system and methods

ABSTRACT

A method for determining a patient&#39;s tooth shade, by generating an electronic image of a patient&#39;s tooth wherein the image includes color information representative of the patient&#39;s tooth shade; storing color information representative of a plurality of tooth shades; and comparing the color information of the image with the stored tooth shade color information to identify one or more tooth shades having a combined color that corresponds to the patient&#39;s tooth shade. the image of the patient&#39;s tooth and the stored tooth shade color information are electronically displayed in color to enable a viewer to comparatively determine the color of the patient&#39;s tooth shade. Preferably, the patient&#39;s tooth shade is determined by selecting portions of the electronically displayed image of the tooth; determining the color of the selected portions; and comparing that color information with the stored tooth shade color information to determine the color of that portion of the patient&#39;s tooth. The selection of portions of the tooth is generally repeated until a tooth shade color is determined for all spatial locations of the image of the patient&#39;s tooth.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of application Ser. No.09/385,615 filed Aug. 30, 1999, now, U.S. Pat. No. 6,132,210, which is acontinuation of application Ser. No. 09/082,030, filed May 18, 1998, nowU.S. Pat. No. 5,961,324, which is a continuation-in-part of applicationSer. No. 08/494,979, filed Jun. 26, 1995, now U.S. Pat. No. 5,766,006.

BACKGROUND ART

[0002] In dentistry, there has been a shift in recent years from aphilosophy of drilling and filling to one of prevention and cosmetics.By way of example, many people today are choosing to have clinicalprocedures done to enhance their smile and appearance. Most of theseprocedures involve the modification of tooth shape, alignment, and/orcolor.

[0003] The use of intraoral video and/or imaging systems (hereinafter“intraoral camera system”) has grown rapidly in dentistry over the pastfew years. Such systems are widely utilized in “show and tell” settings,i.e., where the dentist can show and illustrate particular features of apatient's mouth. These intraoral camera systems are rapidly becoming keycomplex diagnostic and treatment planning. Presently, approximately 30%of the practicing dentists in the age group between about 35-54 own andutilize intraoral camera systems. It is expected that that percentagewill only increase with increased familiarity. See Dental ProceduresReport, pgs. 22-24, February 1995.

[0004] A necessary step in altering a patient's tooth color is todetermine the “shade” of the existing tooth. For example, those personsseeking a whiter, brighter smile are still assessed to establish theirexisting tooth color so that an appropriate before and after comparisoncan be made. Shade determination is even more important for thosepersons seeking reconstructive work, since one goal of thereconstructive process is to achieve a natural appearance. Therefore, itis important to know the existing tooth shade so that it can beaccurately matched with the new restoration.

[0005] The dental profession utilizes standardized shade guides createdby those companies which manufacture the reconstructive materials. Onewell-known shade guide is the Vita™ shade guide, which includes sixteendifferent shades. Other, less popular shade guides include those guidesprovided by Bioform™ and SR-Vivadent™.

[0006] These shade guides are utilized in a rudimentary fashion. Theguide itself is a plastic plate with a plurality of removable color tabsthat are shaped like a tooth, e.g., the front tooth. Typically, toassess a patient's tooth shade, a dentist removes one of the coloredtabs and holds it up to the patient's tooth so that she can “eyeball”the closest match possible. Understandably, there are many variables tothis method, some of which stem from the subjectivity of the dentistmaking the eyeball assessment.

[0007] Once the tooth shade is determined, the information is usedrelative to the particular procedure needed. In bonding or filling atooth, for example, the composite materials required for the restorationare specified within the range of the shade guide, e.g., one of sixteenshades for the Vita™ range. More particularly, if a crown, bridge ordenture is needed, the patient's shade must be determined andcommunicated correctly to the lab that make the crown, bridge ordenture.

[0008] The communication of shade information between the dentist andthe lab is extremely important. Often, there is a break-down or failurein this communication, resulting in a poor shade match for the patient.In some cases, a particular dentist utilizes an uncommon shade guide,thereby leaving the lab technician to eyeball and convert the shadeinformation to a Vita standard shade (since porcelain is often made fromthe Vita™ shade guide). This too can result in improper shade matching.

[0009] The process for selecting the porcelain for a particular toothshade illustrates the difficulty in assessing and manufacturing thecorrect color match. If, for example, a crown of Vita™ shade A3 isdesired, porcelain is built by hand with a paint brush onto a model ofthe tooth to be restored. The porcelain is built in layers on the modelto achieve translucency and natural appearance. Each layer has aparticular color and intensity associated with it. To generate shade A3,the technician follows a “recipe” that is given by the manufacturerVident™, requiring a different shade for each layer of porcelainapplied. If a doctor asks for a shade that is not a Vita™ standardshade, the technician typically seeks to achieve that shade by combiningdifferent porcelain shade combinations together, to increase or decreasethe chroma, hue and value of the shade.

[0010] To further complicate the color-matching process, some dentistsare simply not skilled in taking and determining shade information.Therefore, these dentists sometimes send their patients directly to thelab where the technician can determine the shade information.Alternatively, these dentists sometimes have a technician come to theiroffice. In either event, there is, at times, one more level ofsubjective uncertainty injected into the correct match and determinationof a patient's tooth shade. Thus, there is a need for more improvementsin this area.

SUMMARY OF THE INVENTION

[0011] The invention relates to a method for determining a patient'stooth shade comprising the steps of electronically imaging a patient'stooth to obtain an image which includes color information representativeof tooth color; determining the patient's tooth color from the colorinformation of the image; and displaying the determined tooth color toidentify one or more tooth shades having a combined color correspondingto the determined tooth color.

[0012] Preferably, the displayed data includes RGB chromaticities of thecolor image, and the patient's tooth color is determined by averagingthe color information at selected locations of the image whichcorrespond to different spatial locations of the patient's tooth. Ifdesired, the color information of the patient's tooth can beelectronically stored for use at a later time.

[0013] The identification of tooth shade(s) is generally made bycomparing the determined tooth color to a plurality of electronicallystored tooth shades. If desired, both the identified tooth shade(s) andthe color information of the image of the patient's tooth can beelectronically displayed to enable a visual comparison to be madetherebetween. Preferably, information which includes the identifiedtooth shade(s) can be communicated to a laboratory for use inmanufacturing a reconstructed tooth for the patient. This is typicallydone by painting one or more porcelain layers onto a model of thepatient's tooth, each of the layers of porcelain corresponding to theidentified tooth shade(s) or a color fraction thereof.

[0014] For optimum results, either the patient's tooth or a tooth shadeis imaged at a controlled, reproducible distance, and a reference colorto assist in determining the patient's tooth color from the colorinformation of the image. Also, the patient's teeth are cosmeticallywhitened before conducting the electronic imaging step.

[0015] Another aspect of the present invention relates to a method fordetermining a patient's tooth shade comprising the steps ofelectronically imaging a patient's tooth to obtain an image whichincludes color information representative of tooth color; determiningthe patient's tooth color from the color information of the image; andidentifying one or more tooth shades having a combined colorcorresponding to the determined tooth color. The patient's tooth colorcan be determined by electronically displaying the color information ofthe image and by averaging the color information at selected locationsof the image which correspond to different spatial locations of thepatient's tooth.

[0016] The method may include the further step printing theelectronically displayed color information of the image to produce arecord or electronically storing the color information of the patient'stooth for use at a later time. Advantageously, the identification oftooth shade(s) is made by comparing the determined tooth color to aplurality of electronically stored tooth shades, and both the identifiedtooth shade(s) and the color information of the image of the patient'stooth can be electronically displayed to enable a visual comparison tobe made therebetween. Information regarding the identified toothshade(s) can be communicated to a laboratory for use in manufacturing areconstructed tooth for the patient. The reconstruction is performed bypainting one or more porcelain layers onto a model of the patient'stooth, each of the layers of porcelain corresponding to the identifiedtooth shade(s) or a color fraction thereof.

[0017] In order to optimize the color determining step, one of thepatient's tooth or a tooth shade or both can be imaged at a controlled,reproducible distance. The step of isolating the patient's tooth fromstray light can be used to enhance the image. Also, a reference colorcan be provided to assist in determining the patient's tooth color fromthe color information of the image. When used, it is helpful to surroundthe patient's tooth with the reference color before the imaging step.Often, the patient's tooth is cosmetically whitened before conductingthe electronic imaging step.

[0018] The invention also relates to a system for determining the toothshade of a patient's tooth, comprising a camera for capturing an imageof the patient's tooth, the image including color informationrepresentative of tooth color; and a shade analyzer sub-system whichelectrically communicates with the camera. The shade analyzer typicallyincludes a color processor for determining the color of the patient'stooth from the color information of the image; and a color correlationsection for evaluating the color of the patient's tooth and foridentifying a tooth shade having a color corresponding to the color ofthe patient's tooth. Preferably, the system further comprises anelectronic storage medium for storing shade information representativeof a plurality of tooth shades, each tooth shade corresponding to adifferent tooth color.

[0019] The camera is preferably a color CCD camera comprising an arrayincluding a plurality of proximately located pixels. Generally, thecamera comprises three CCD arrays, each of the arrays collecting imagedata corresponding to red, green or blue, and the subsystem furthercomprises a section for integrating at least a portion of the tooth todetermine an average color over that portion of the tooth by combiningcolor fractions of known tooth shades. Thus, the color correlationsection should include sufficient memory for associating the color ofthe patient's tooth with one or more known tooth shades.

[0020] Advantageously, the system further comprises a monitor forsimultaneously displaying color information of the tooth and tooth shadeinformation so that visual comparisons can be made. Also, a printer canbe used for printing the color information displayed on the monitor forcommunicating this information to a laboratory for reconstruction of thepatient's tooth.

[0021] The color processor preferably includes electronic circuitrywhich determines RGB chromaticities of the color image, each of theplurality of tooth shades has a corresponding RBG representation of thetooth color, and the electronic circuitry quantitatively compares theRGB chromaticities with the RGB representation of each of the toothshades, thus comparing or identifying a tooth shade based upon acomparison of the chromaticities and the RGB representations. Foroptimum results, the system further comprises a sample box (or isolationsleeve) for isolating the tooth from stray light. This isolation sleevepreferably includes a first compartment that contains a reference colorand a second compartment for receiving the patient's tooth or areference shade. It also may have a side wall of a length which assistsin obtaining reproducible results by positioning the camera at acontrolled distance from the patient's tooth or the tooth shade. Evenwithout the isolation sleeve, the system may include a reference cardhaving a background color to enhance the image of the patient's tooth,the card optionally including an opening for receiving the patient'stooth or a reference shade.

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] A more complete understanding of the invention may be obtained byreference to the drawings, which illustrate preferred embodiments inwhich:

[0023]FIG. 1 shows a prior art intraoral camera system;

[0024]FIG. 2 illustrates a tooth shade analyzer system constructedaccording to the invention;

[0025]FIG. 3 illustrates a typical prior art tooth shade guide;

[0026]FIG. 3A illustrates one tooth shade tab of the shade guide in FIG.3;

[0027]FIG. 4 illustrates a digital data block constructed according tothe invention for specifying a tooth shade color, color imageinformation, and an associated manufacturer;

[0028]FIG. 5 shows an operational use of the system of FIG. 2;

[0029]FIG. 6 illustrates image averaging and specification techniques,according to the invention;

[0030]FIG. 7 illustrates an image comparison on the system of FIG. 2;

[0031]FIG. 8 illustrates a common aperature color CCD camera andhandpiece constructed according to the invention, and which form anotherembodiment of a shade analyzer system;

[0032]FIG. 9 shows an alternative CCD array technology, according to theinvention, and which is suitable for use within the camera of FIG. 8;

[0033]FIG. 10 is a perspective view of a background color referencemember;

[0034]FIG. 11 is a side schematic view of a camera and isolation sleeveaccording to the invention; and

[0035]FIG. 12 is a perspective view of the isolation sleeve of FIG. 11viewed from the camera when detached therefrom.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] A number of different aspects of the invention are disclosed. Inone aspect, the invention provides a system for determining the toothshade of a patient's tooth. A camera, preferably an intraoral camera,captures the image of the patient's tooth, including color informationrepresentative of the tooth's color. A shade analyzer sub-system is inelectrical communication with the intraoral camera, and preferably has(i) a color processing section for determining the color of thepatient's tooth from the color information of the image; (ii) storagememory for storing shade information representative of a plurality oftooth shades (i.e., each of the tooth shades corresponds to a differenttooth color), (iii) a color correlation section for comparing the colorof the patient's tooth to the plurality of tooth shades and foridentifying a tooth shade with a color corresponding to the color of thepatient's tooth, and (iv) means such as a display terminal forcommunicating the identified tooth shade to a user of the system.

[0037] In another aspect, the system includes a monitor used to displaythe color image to a user of the system. The shade analyzer sub-systemthus communicates a tooth color representative of the identified toothshade to the monitor, thereby providing a user of the system with avisual comparison of the patient's tooth color with the color of theidentified tooth shade.

[0038] In a further aspect, the color processing section determines RGBchromaticities of the color image, and each of the plurality of toothshades has a corresponding RGB representation of the tooth color storedin memory. In such an aspect, the color correlation sectionquantitatively compares the RGB chromaticities with the RGBrepresentation of each of the tooth shades, and compares and identifiesa tooth shade based upon an comparison of the chromaticities and the RGBrepresentations.

[0039] In still another aspect of the invention, the system associatesone or more of the plurality of tooth shades with at least onestandardized shade corresponding to a particular manufacturer'sporcelain product. Accordingly, the system communicates the standardizedshade to a user of the system so that the desired tooth may beconstructed.

[0040] The color correlation section of the invention can also identifya combination of tooth shades having a combined color corresponding tothe color of the patient's tooth. As such, the system can (i) associateeach of the tooth shades in the combination with at least onestandardized shade corresponding to an particular manufacturer'sporcelain product, (ii) specify fractions of each of the standardizedshades needed to form the combined color, and (iii) communicateinformation identifying the fractions of each of the tooth shades in thecombination to a user of the system.

[0041] The invention also provides a tooth shade analyzer system fordetermining the tooth shade of a patient's tooth. One section of theanalyzer communicates with an intraoral camera of the type whichcaptures the image of the patient's tooth, including color informationrepresentative of a color of the tooth. Other sections of the analyzer(i) determine the color of the patient's tooth from the colorinformation of the image, and/or (ii) store shade informationrepresentative of a plurality of tooth shades. A color correlationsection compares the color of the patient's tooth to the plurality oftooth shades and identifies one or more tooth shades with a combinedcorresponding to the color of the patient's tooth. Finally, the systemincludes means for communicating with devices such as the displaymonitors to communicate the one or more identified tooth shades to auser of the system. In this manner, the analyzer matches the patient'stooth with one or more corresponding tooth shades and assists inreconstructing the patient's tooth.

[0042] The invention also includes a method for determining the toothshade of a patient's tooth, including the step of: capturing an image ofthe patient's tooth with a camera, the image including color informationrepresentative of tooth color; determining the color of the patient'stooth from the color information of the image; comparing the color ofthe patient's tooth to a plurality of tooth shades, the tooth shadesbeing stored in an electronic medium; identifying one or more toothshades with a combined color corresponding to the color of a patient'stooth; and communicating the one or more identified tooth shades to auser of the system.

[0043] A further method of the invention compares to tooth shade of apatient's tooth after the patient's teeth are cosmetically whitened,including the steps of: capturing a first image of the patient's toothwith an intraoral camera before the tooth is cosmetically whitened, thefirst image including first color information representative of a colorof the tooth; processing the color information of the image to determinethe color of the patient's tooth; comparing the color of the patient'stooth to a plurality of tooth shades, the tooth shades being stored inan electronic medium; identifying one or more tooth shades with acombined color corresponding to the color of the patient's tooth;whitening the teeth; communicating the one or more identified toothshades to a user of the system; and viewing the patient's tooth on amonitor after the whitening step while simultaneously displaying animage of the patient's pre-whitened tooth, to provide before and afterimagery.

[0044] In another aspect of the invention, a process is provided formanufacturing a reconstructive tooth for a patient, including the stepof: capturing the image of the patient's tooth with an intraoral camera,the image including color information representative of a color of thetooth; processing the color information of the image to determine thecolor of the patient's tooth; comparing the color of the patient's toothto a plurality of tooth shades, the tooth shades being stored in anelectronic medium; specifying one or more tooth shades, and anyfractions thereof, having a combined color corresponding to the color ofthe patient's tooth; and painting one or more layers of porcelain onto amodel of the patient's tooth, each of the layers of porcelaincorresponding to the specified tooth shades and the fractions thereof.

[0045] The invention also includes a system for determining the toothshade of a patient's tooth by utilizing color CCD cameras. In oneaspect, a color CCD camera captures the image of the patient's tooth,including color information representative of a color of the tooth. Ashade analyzer sub-system is connected for electrical communication withthe CCD camera, and has (i) a color processing section for determiningthe color of the patient's tooth from the color information of theimage, (ii) a storage section for storing shade informationrepresentative of a plurality of tooth shades, each of the tooth shadescorresponding to a different tooth color, (iii) a color correlationsection for comparing the color of the patient's tooth to the pluralityof tooth shades and for identifying a tooth shade with a colorcorresponding to the color of the patient's tooth, and (iv) a sectionfor communicating the identified tooth shade to a user of the system.

[0046] In accord with the invention, the CCD camera can include threeCCD arrays, each of the arrays collecting image data corresponding to acolor selected from the group of red, green and blue. Alternatively, theCCD camera can include a single CCD array, including a plurality ofproximately located pixels corresponding to a color selected from thegroup of red, green and blue, each of any group of three pixels having adifferent color associated therewith.

[0047]FIG. 1 illustrates a typical prior art intraoral camera system 10.The system 10 includes a wall-mounted intraoral camera 12, whichincludes twin halogen lamp light source (not shown) for uniformilluminance, one or more handpieces 14 a, 14 b, a digital videoprocessor 16 (typically an IBM-compatible PC), which includes a harddrive 18 and floppy drive 20, and a color video monitor 22.

[0048] In operation, a dentist points a selected handpiece 14 a, 14 b atthe target location within the patient's mouth to illuminate and viewthe resulting, full color image on the screen of a monitor 22. Theendoscope handpieces 14 a, 14 b come in varying styles, including a wideangle configuration, e.g., a 100 degree field of view (FOV) forposterior and anterior views, and a near 0 degree FOV for a full-archand full face images. The processor 16 provides storage for any selectedimage, and can further display selected close-ups on the monitor screen22 through zoom capabilities.

[0049] There are several manufacturers of intraoral camera systems,offering an array of features. For example, Insight™, of San Carlos,Calif., offers a Power 0/100 similar to the one shown in FIG. 1. Othermanufacturers include Cygnus Instruments, Inc. (Cygnascope™), of Goleta,Calif. VistaCam™ is yet another prior art intraoral camera system thatincorporates a 90-degree FOV fiberoptic handpiece that delivers fullcolor images from about 6 mm, i.e., the size of one typical tooth, to animage of the patient's whole smile. Therefore, the intraoral camerasystems described above offer full color imagery of a patient's tooth.The image may be stored, e.g., in a personal computer on a disk 18, 20.Also, the images can be advantageously displayed on the monitor screen22.

[0050] The invention makes use of an intraoral camera system of the typedescribed above. Specifically, FIG. 2 illustrates one embodiment of theinvention and which shows a tooth shade analyzer 29 including anintraoral camera 30, associated handpieces 32 a, 32 b, and ashade-analyzer subsystem 34. Preferably, the subsystem 34 is a digitalvideo processor that is similar to the processor 16 of FIG. 1, and thuspreferably includes the video processing capabilities of the processor16, such as known to those skilled in the art. A storage medium 36, suchas a hard disk 18 or floppy 20, stores digital color images of aplurality of tooth shades, such as each of the sixteen shades of theVita™ shade guide. The storage medium 36 further stores any imagescollected by a user of the system 29.

[0051] The storage of the digital color images of one or more shadeguides is relatively straight-forward. In particular, FIG. 3 illustratesone shade guide 38 having sixteen separate tabs 40. Each of the tabs 40is removable from the guide 38 so that it can be placed and viewed nextto the patient's teeth. FIG. 3A illustrates one tab 40 that is removedfrom the guide 38. The tab holder 40 a typically includes colorinformation about the selected tab 40, here shown as shade “A2.”

[0052] In accordance with the invention, each of the tabs 40 areilluminated by one or the handpieces 32 a or 32 b so that a color imageis captured and stored in the medium 36. The image is accompanied byassociated information about that shade, such as the manufacturer type,e.g., the Vita shade guide, and the particular shade, e.g., “A2.” Forexample, FIG. 4 illustrates one storage block 42 of data for storage asdigital memory and which is representative of one tooth shade. Inparticular, block 42 a includes digital color image information, block42 b denotes the particular tooth shade, e.g., “A2,” and block 42 cdenotes the manufacturer name, so that the associated tooth shadeporcelain can be purchased from the correct manufacturer. Generally, theblocks 42 b, 42 c of data are represented by digital words that specifythe information, while block 42 a includes image data corresponding toboth color and spatial information.

[0053] Preferably, the color image data within the block 42 a is in aRGB format (i.e., Red, Green, Blue digital format, such as known tothose skilled in the art) which specifies a color pixel within the imageto an accuracy exceeding the number and color spread of the selectedshade guides. For example, if sixteen shades are stored within thememory 36, then the bit-specificity required of the color-coded datawithin the image block 42 a should span and discriminate at least thosesixteen shades of white. For example, if each color of the RGB isspecified at 3-bits each, then 256 standard colors are discernible bythe system 29 to cover sixteen tooth shades. Greater color determinationaccuracy is achieved with increased color-coding. In addition, if allthe manufacturers of tooth shades are loaded into the memory 36, theneven more color-coded accuracy is required, e.g., 8-bit per RGB color.

[0054] The display of the color imagery within the digital data block 42a is known to those skilled in the art. Specifically, the RGBinformation within the data block 42 a specifies the color for eachpixel on the monitor 44, FIG. 2, which is similar in capability to themonitor 22 of FIG. 1. The particular video driver (not shown) used todrive the monitor 44 specifies and controls the color and image displayas appropriate. The RGB format specifies a color corresponding tosignals which produce a suitable color picture on the monitor having thereference colors defined by the RGB chromaticities.

[0055] Once the plurality of shades are loaded into memory 36, thesystem 29 is ready for use. FIG. 5 illustrates this process in moredetail. A user illuminates the desired tooth 50, (illustratively shownoutside of the patient's mouth) with the handpiece 32 a such that animage 52 of the tooth 50 is displayed on the monitor 44. Although thisstep of forming an image on the screen 44 is not required, it helps toassess the accuracy to which the camera 30 captured the image 52.

[0056] As above, this image data 52 is stored into a block such as block42, FIG. 4. The shade analyzer-subsystem then compares the digitalinformation within the block of data representing the image 52 with theblocks of data representative of the several tooth shades. In oneembodiment, a comparison of the RGB values is made between the toothimage data 52 and the several shades to find a best fit or match.

[0057] It is important to note that this process has several advantagesover the prior art. In particular, as long as the same camera 30captures the data of the tooth shades and the patient's tooth shadedata, it does not matter that the stored image color within theprocessor 34 match the perceived color or hue observed by a typicalhuman. That is, as long as the data captured by the system 29 correlatesto the same reference, e.g., the same camera 30, the match between theimage 52 and the plurality of tooth shades will be calibratedautomatically. This is in contrast to many of the prior art references,e.g., U.S. Pat. No. 5,383,020, which requires a sensitivity maxima ofthe human eye. The invention has no corresponding limitation because thecamera of the system 29 objectively gathers the data from the sameinternal source. Nevertheless, it is preferable that the color displayand coding of the camera 30 correspond to natural and perceived colorsso that the image appears normal to a viewer.

[0058] Those skilled in the art should appreciate that the data from theplurality of tooth shades can also be loaded from a floppy disk into thesubsystem 34. More particularly, the tooth shade data from the one ormore manufacturers can be installed directly into the system 29 withoutphysically capturing the image of each tooth shade, provided that thedata is captured by a camera that is similar to the camera 30, or by acamera that is calibrated to within an acceptable margin to the camera30. In this manner, a user of the system 29, e.g., a dentist, need neverhave actual tooth shade tabs at the office. Rather, the system 29 isused to capture color data on the patient's tooth; and the stored toothshade information within the system 29 is automatically extracted,including a manufacturer identifier.

[0059] It is worth noting that not all tooth shade information isreadily derived from the patient. In particular, unlike the tooth shadesfrom the shade guides, which are very uniform in color, a patient'stooth can include a myriad of different shades. Tooth stains and thelike alter the tooth's color spatially, so that a selection orintegration of color is preferred. FIG. 6 illustrates one embodiment ofthe invention for dealing with this non-uniformity.

[0060] Specifically, FIG. 6 shows a tooth image 60 of a patient's toothon the monitor 44. Each pixel 62, which is grossly over-sized forillustrative purposes, corresponds to a different spatial location andcolor of the tooth's image 60. Accordingly, a user of the system 29 canselect one of the pixels at the desires color by pointing and clicking amouse pointer 64, via the mouse 66, at any selected location (thedetails of the mouse and mouse pointer are widely known to those skilledin the art without further reference hereto, and are shown forillustrative purposes only). This selected information is then storedwith the block of data, e.g., block 42 b of FIG. 4.

[0061] Alternatively, the image of the tooth 60 can be averaged over aselected area by selecting a particular region for which the colorimagery is averaged. For example, if the tooth 60 is to be averaged overthe two pixels identified by the outline 68, then the subsystem 29averages the two RGB values and specifies the tooth shade match in block42 b as an average of the two.

[0062]FIG. 7 illustrates another monitor 44′ and subsystem 34′constructed according to the invention, each of which is similar to themonitor 44 and subsystem 34 of FIG. 2, respectively. In the illustratedembodiment, an image 70 of a patient's tooth is matched to theappropriate shade, as above, and then an image 72 of the matched shadeis also displayed on the monitor 44′. The subsystem 34′ furthergenerates and displays the tooth shade identifying match, e.g., “A2,” onthe screen 44′ so that the ordering information about the toothreconstruction is easily ascertained. In the illustrated embodiment, avisual comparison between the tooth and the selected shade is madeavailable to both the dentist and the patient. The display 44′ can alsodisplay the particular porcelain product information.

[0063] If should be apparent to those skilled in the art that the dataand information displayed on the monitor 44′ can also be downloaded to aprinter 76, so that a permanent record of the exam is obtained inhard-copy form.

[0064] In the event that the system 29 of FIG. 2 does not find a closematch to one existing tooth shade, the subsystem 34 specifies acombination of tooth shades that correspond to the color of thepatient's tooth image data, e.g., the image 52 of FIG. 5. Preferably,this information is determined in fractions of the appropriate shade,e.g., ½ “A2” and ½ “C4.” Such fractions are determinable, according toone embodiment of the invention, by comparing the RGB data within theplurality of tooth shade blocks 42 b with the actual patient's toothshade image information. If the RGB data of the patient's tooth is equalto the chromaticity sum of ½ “A2” and ½ “C4,” then such a fraction isentered into the match data block 42 b for the selected tooth shade, anddisplayed on the screen 44 for the user. The chromaticity mathematicsused to combine and subtract colors is known to those skilled in theart. Further detail may be found with reference to “TelevisionEngineering Handbook,” edited by K. Blair Benson, McGraw-Hill (1986),which is incorporated herein by reference.

[0065] The patient's tooth shade information is typically communicatedto a laboratory which manufactures the reconstructed tooth via aplurality of porcelain coatings. This process of constructing porcelainlayers onto a tooth model is known in the art; although thespecification of the differing porcelain layers by data generated by anintraoral camera is specific to the invention. In the event that certainfractions of different porcelain layers are needed, such as describedabove, the system of the invention again provides and generates theappropriate shade fractions corresponding to the multiple layers.

[0066] Other color theory details, including the adding and subtractingof multiple colors, may be found with reference to the followingpatents, each of which is expressly incorporated herein by reference:U.S. Pat. No. 5,383,020, entitled “Method and apparatus for determiningthe color of a translucent object such as a tooth;” WO 86/03292,entitled “A spectrum-photometer device for exactly determining the colorof a dental plate and or dental pluggings;” U.S. Pat. No. 3,986,777entitled “Tristimulus calorimeter for use in the fabrication ofartificial teeth;” U.S. Pat. No. 4,654,794 entitled “Methods fordetermining the proper coloring for a tooth replica;” U.S. Pat. No.4,836,674 entitled “Method and apparatus for determining color, inparticular of a dental prosthesis;” U.S. Pat. No. 5,231,472 entitled“Color matching and characterization of surface coatings;” and U.S. Pat.No. 4,247,202 entitled “Automatic computing color meter.”

[0067]FIG. 8 illustrates another embodiment of the invention, and whichincludes a common aperature color CCD camera 80. The camera 80 iscommonly known as a “threechip” color camera because of the threeseparate area CCD arrays (and preamplifiers) 82 a, 82 b, 82 c. A complexprism 83 slits the light energy entering through the aperture lenses 84a, 84 b into the three RGB color components. That is, each of the arrays82 uniquely corresponds to one of the three color spectrums, such asRed. A housing 86 surrounds and protects the camera 80 from contact withexternal influences. The outputs from the three arrays 82 a, 82 b, 82 cconnect to a common RS232 interface 88, which is in turn connected, viasignal lines 89, to a digital video processor shade analyzer subsystem,such as described above.

[0068] The size of the camera 80 is small, typically about 40 mm³ orless. One manufacturer of a camera 80 includes Richter Enterprises, ofDel Norte, Colo., which makes the Model AD01 Common Aperture Camera.

[0069] The front aperture section 90 of the camera 80 connects to ahandpiece 92—similar to the handpieces 32 a, 32 b described above—whichis utilized by the dentist to illuminate and capture light imagerywithin the patient's mouth. For example, one acceptable handpiece 92 isan endoscopic handpiece which derives illumination from an externalsource 94, e.g., a lamp. The source is imaged through the endoscope toprovide illumination 96 at the target area designated by the dentist;and the endoscope recaptures the light scattered from within thepatient's mouth to form an image, in conjunction with the camera 80, atthe several CCD arrays 82 a, 82 b, 82 c.

[0070] Each of the CCDs 82 a, 82 b, 82 c is coaligned with the other sothat three distinct pixels, i.e., one each corresponding to R, G or Bchromaticities, substantially view the same target point. Typically,each of the pixels specifies an 8-bit (or even 10-bit) color, so that,in total, the combination of the three arrays form a possible256×256×256 colors. This provides sufficient resolution to discernseveral shades of white within the shade guides, so that a particularguide may be matched with a patient's tooth.

[0071] Alternatively, the arrays of FIG. 8 can be replaced with a singlearray 100, FIG. 9 (as such, the prism 83 is also no longer required).Such an array 100 is popular in lower cost Camcorder technologies. Inone embodiment, the array 100 has serial RGB pixels along each two ofthe array, here denoted as “R”, “G” and “B” within each pixel 102. Thisscheme reduces resolution; although it also reduces costs. As above,each of the three RGB pixels is utilized to assess and determine shadecolor.

EXAMPLE

[0072] The following example illustrates the use of a CCD camera toprovide a dental color match for a patient's tooth. The camera wasconnected to a Personal Computer (“PC”). Several images of the patient'stooth were taken and compared for consistency and definition of colorvariance. To verify consistency, a software program was used tointerface with the camera to take snapshot images of Vita Limun VacuumShades using the camera. The images were be stored on the PC for lateranalysis. All images included the sample media and color reference atthe same location in the frame. A software program was written toanalyze the stored snapshots. Evaluation areas (sample and reference)were sampled and results of the color analysis were presented. The colorreference was used to overcome the auto-brightness which could not bedisabled on the camera. The software automatically adjusted the sampleresults using the color reference.

[0073] The following camera was used and found to be preferred:

[0074] Type: Intraoral

[0075] Manufacturer: Welch-Allyn

[0076] DLS 400 Light source (Sn: D97-652920)

[0077] DHP Hand Piece

[0078] Lenses Green (General purpose), White (Close Focus) and Red (FullFace)

[0079] This camera has a built in light source with a color temperatureof 5600K and was select to provide more consistent results than using aseparate light source. The camera has auto white balance that is turnedoff when using the light source. The camera has no external control ofauto-brightness, but it is controllable by the camera firmware.

[0080] An Image Acquisition Hardware (Frame Grabber) that was used wasan Alaris Quick Video Transport. This results in a sampled color of 24bits being stored on the PC. It has an S-Video (SVHS) input to match theoutput from the camera to provide the best possible results by using thebest signal output from the camera.

[0081] The software uses a Twain protocol interface to grab images fromthe camera using the Frame Grabber. This is a standard softwareinterface that allow other cameras and Frame Grabbers to be testedwithout the need for a change of software.

[0082] The camera was used to provide an image of the sample area of apatient's tooth on a monitor screen. The PC and frame grabber capturedthe image onto the hard disk for analysis. The images stored from theFrame Grabber were selected to be 640 pixels wide by 480 pixels high,using 24 bits per pixel for color, with 8 bits each for red, green andblue (“RGB”). The color is coded as red, green and blue signals, eachwith a value from 0 to 255. An RGB value of (0,0,0) represents black,and (255,255,255) represents white.

[0083] As the RGB values alone are not very intuitive for analyzingcolor, they were converted into the Hue Saturation and Intensity colorspace by using an algorithm. The Gonzalez and Woods method is used tocalculate Hue Saturation and Intensity (HSI), as follows:

[0084] R=Red value for pixel

[0085] G=Green value for pixel

[0086] B=Blue value for pixel

[0087] Intensity=⅓(R+G+B)

[0088] Saturation=1−(3/R+G+B))*Min(R, G, B)

[0089] Hue=Cos⁻¹((0.5*((R−G)+(R−B)))/((R−B)*(G−B))^(0.5))

[0090] If S=0, Hue is meaningless

[0091] If B3/Intensity)>(G/Intensity) then Hue=360−Hue

[0092] Since Hue is an angle in degrees values were normalized to 0.1with Hue=Hue/360

[0093] The RGB color space is a simple cube, with R, G and B emanatingfrom one corner along three perpendicular edges. The origin corner(0,0,0) is black, and the opposite corner (1,1,1) is white. All pointsalong this line from corner to corner are shades of grey. The HSI colorspace is this same cube stood on the origin comer, with the Black Whiteline being vertical. The black-white line is the intensity axis, the hueis given by an angle from the intensity axis and the saturation is thedistance from the intensity axis to the color point (i.e., the radius).The new VITAPAN 3D-Master Shade system uses an L*a*b* Color Sphere todetermine tooth shades based on Value, Chroma and Hue. It is possible toconvert the RGB values to this color system, if necessary or desired.

[0094] The software for analyzing the image was written to allow theselection of a particular area of the tooth for analysis. This area wasactually selected by human eye, and was a matter of judgement.

[0095] The initial tests using a normal video camera and desktopcomputer camera showed that the white color balance used by thesecameras (and auto brightness) changed the colors of the subject viewedby the cameras even while trying to maintain constant conditions. Onmost cameras it is not possible to turn off the auto white balance.

[0096] The initial tests with the Welch-Allyn intraoral camera wereperformed by using the camera hand held, as this reproduces the currentmethod used by a dentist to take such pictures. These test gave picturesof varying quality, orientation, distance from tooth etc. The comparisonof the data taken from these pictures indicated that the camera candetermine color differences, but that the auto brightness on the cameracould cause problems.

[0097] A3, B3 and C3 Vita Shade Images were imaged by the hand heldcamera. The shades were still in their holder, and the camera passedover each one to capture the images. The software program was used tosample the images, but it was found that intensity values across thedifferent sample sets varied greatly. The lighter shades (A1, B1, C1,and D2) gave the poorest cross sample correlation, possibly due to thesample color being closer to white with ambient lighting and reflectioncausing interferences.

[0098] The inclusion of a reference color around the sample helped togive a reference for intensity correction, but the varying lens distancefrom sample still gave images of varying quality. The use of a referencecolor was an attempt to simultaneously obtain distance measurement,color reference and stray light protection. As shown in FIG. 10, abackground color reference member 10 was a strip of construction paper15 with a cut-out or opening 20 that approximates the shape of thepatient's tooth or a reference such as a Vita shade. Each image wastaken at approximately the same distance and orientation. Better resultswere obtained than without the reference, but there was still a problemwith the Hue, and the varying distance of the camera, and hence lightsource from the sample. Varying angle on the reference also madedifferences. When measuring the color of the tooth or reference shade,the sampled area 14 should be chosen to avoid lamp reflections 16 orother inconsistencies. The background 18 of the reference member is alsosampled.

[0099] The final preferred method used a single fixed area of the imagefor sample and a separate area for reference, with the image size andorientation controlled by the use of a sample box or isolation sleeve. Acream color was chosen for the sleeve as it could also be used as thereference color for all tests. The isolation sleeve was attached firmlyto the camera head to allow the light source to illuminate the inside ofthe sleeve and to enable the camera lens to have an unobstructed view ofthe bottom inside face of the sleeve and the sample opening.

[0100] As shown in FIG. 11, the isolation sleeve 25 surrounds the cameralens 30, and has an opening 33 in the end opposite to the lens toaccommodate the Vita shade or the patient's tooth 60 to be sampled. Theisolation sleeve 25 reduced two major consistency problems encounteredwhen using the camera without it. Firstly, it reduced the variation indistance and orientation of the camera 55 from the subject, andsecondly, it removed unwanted ambient light contamination. As shown inFIG. 12, the inside surfaces 40 of the isolation sleeve 25 were madematt black to avoid reflection and prevent color contamination of thesubject. The color reference 35 was provided in a separate chamber inthe box to prevent cross color contamination with the tooth or sampleshade. A double isolation sleeve 25 was used with both chambers 45, 50in front of the light source and camera lens, but with the subject andthe color reference in separate chambers. Half of the resultant imagewas the color reference, while the other half was of the tooth to beevaluated.

[0101] Only one area was sampled on the subject tooth, although it ispreferable to divide the subject into multiple target areas, andconstruct a shade database for each sub-section. Thus, when a patient'stooth color partially matches two shades, the areas can be referencedindividually.

[0102] Where the sample area contains a reflection of the light source,this greatly affects the averaged values for the colors. The lampreflection is usually samples as very near bright white (255,255,255)).To help omit these points and any dark spots on the sample, a selectivepixel usage was adopted. The area being sampled was averaged, and theMaximum and Minimum RGB values noted. The area was then rescanned andany pixel with any of the RGB values outside of half way from theAverage to Max value or Average to Min value are ignored, i.e.:

(Min+Average)/2)<Valid Value<(Max+Average)/2

[0103] When the color references were compared, it was found that asdifferent shades were viewed, the saturation, intensity and hue of thereference also changed. This was most likely due to reflection from thesample contaminating the color reference, and the fact that the cameraautomatically adjusted for brightness. Hence the greatest change is seenin intensity and saturation.

[0104] The values for the color reference were used to adjust the valuestaken from the sample in the same image. The difference in the valuefrom the color reference for each picture was lineally adjusted by adelta value such that the adjusted values were then identical for eachimage. This delta value for each image was also applied to the valuestaken for the sample. This made the comparison of one image to anothermuch more consistent. The hue variation in the reference sample wasfound to be less than 1.5% of the camera's full range.

[0105] While the intensity comparison of the samples exhibited avariation in the shades, there is still variation in the levels from oneset of samples to another i.e., the different images of the sample shadegive different results. Although the saturation varied over a largerrange of values, the difference between the sample sets for the sameshade was still noticeable.

[0106] The hue of the sample taken from the images was erratic andshowed very poor correlation between the same shade in the differentsets of samples. To make the images more consistent, the image isadjusted by applying a delta to the sampled RGB value that would makethe reference color become the same value for each image. Thiscompensates for the auto brightness control of the camera. When thereference adjustments were taken into account, the correlation of theresults improved. The RGB reference values were set to R: 0.66583, G:0.57465, and B: 0.55137. A reflection filter was also inserted into thesampling algorithm. This attempts to ignore any pixel that is more than50% away from the average value. The correlation between sample setsimproved with these adjustments, and with reflection filtering.

[0107] The results showed that the use of a normal CCD camera with anexternal light source is possible, but would require a more extensivecalibration system than is required by using an internal light source.The color temperature of the light being seen by the camera would varydepending on the location of the patient near the light source, amountof sunlight experienced on the day, and on the type of room lightingused. Fluorescent lighting gives a flickering light, which can causeproblems with certain shutter speeds on the camera.

[0108] The camera must also be able to deselect the auto white balancewhich would distort colors in favor of a balanced picture, and ifpossible the auto brightness control which would change intensity. Thiswould give rise to much manual adjustment as the ambient lightingconditions change to ensure the camera was operating non-saturated.

[0109] An intraoral camera with a built in light source that isdelivered to the sample from the camera head is therefore preferred. Thefirst observation from using the camera was the inaccuracy and poorrepeatability of trying to sample a shade using the camera held in thehand only. The distance from the sample seems to be critical, and theambient light contaminating the sample caused poor readings. Therefore,it is preferred that the ambient light be prevented from contaminatingthe sample, and that a distance reference from the camera lens to thesample be provided. The initial trial with the sample box attached tothe camera head resolved these problems and provided good results.

[0110] It is useful to avoid the introduction of stray light as thecolor reference is affected by color contamination from light reflectedfrom the sample. When the color reference values were used to create acorrection factor for the RGB values obtained from the sample, theisolation sleeve 25 minimizes these problems by reducing the lightreflected from the sample and the light contamination of the colorreference.

[0111] The isolation sleeve 25 is attached to the camera head to preventstray light from entering. The sleeve is shown viewed from the cameraside. The orientation is dependent on the light source, as it ispreferred to have the light distributed evenly into both chambers of thesleeve. The black walls of the isolation sleeve can cause interferenceswith the auto brightness control of the camera, as the auto brightnesscontrol will try to maintain an ‘average’ brightness for the whole imageand thus may cause the actual sample or color reference to saturate.This is where the camera auto adjusts to the point that the samplereturns a color value of pure white to balance the black in the rest ofthe image, and all shades of the white, and thus the real color will belost. The auto brightness should be turned off for best results.

[0112] The color reference in the isolation sleeve can be madeconsistently, so that no further calibration should be required. Eachimage will calibrate itself. If the color reference is not sufficientlyconsistent, the reference itself will need calibration using a furthercolor reference (a Vita shade for example). If the reference fades withuse, it can be replaced to insure reference consistency.

[0113] If the Colour Space of the Lumin Vacuum Shade Guide is not evenlydistributed, the current system of first selecting the hue, and then thesaturation and lightness makes the analysis for the software difficultif the camera cannot accurately determine the exact hue. The camera isbeing used to determine hue differences over a very small range of itsoperative area (less than 1.5% of the full camera range). The resolutionrequired to perform this analysis should be available on the camera, anddiscernible on the PC. As this is a relatively small range of values,the repeatability of the image taking process must be made very high. Itis not just a case of taking a picture of the tooth and analyzing it.The conditions required by the camera must be stabilized. Again, theisolation sleeve provides a controlled environment such that repeatableand reproducible images are obtained. The images contain the colorreference, and can if necessary be sent to the laboratory together withthe dentist's requirements for reconstructing the tooth. For infectioncontrol, the isolation sleeve should be made to be disposable.

[0114] Older shade guides are not evenly spaced and this can introduceinconsistencies in the results. Newer 3D color shades are more evenlydistributed and thus make color differentiation easier for the camera.The new shades include 5 numbered lightness (value) categories. Theseare then subdivided into a choice of three Chroma levels (1, 2, or 3)and for lightness groups 2, 3 and 4 there are three hues (L, M and R).This requires a different approach to analyzing the sampled data, aspriority is now given to the Lightness value (Intensity) and then theChroma (Saturation). It was found that both values gave good correlationacross different data sets. The Hue is then only divided into threepossible categories (depending on the lightness category) which makesthe color analysis much more consistent. The sample box also providesbetter color and hue correction as well as intensity correction forthese new shades.

[0115] Further details of these tests can be found in the report that isannexed to U.S. Pat. No. 5,9961,324. It should be apparent to thoseskilled in the art that certain modifications can be made to theinvention herein without departing from the true scope of the invention.

What is claimed is:
 1. A method for determining a patient's tooth shade,which comprises: generating an electronic image of a patient's toothwherein the image includes color information representative of thepatient's tooth shade; storing color information representative of aplurality of tooth shades; and comparing the color information of theimage with the stored tooth shade color information to identify one ormore tooth shades having a combined color that corresponds to thepatient's tooth shade.
 2. The method of claim 1 wherein the image of thepatient's tooth and the stored tooth shade color information areelectronically displayed in color to enable a viewer to comparativelydetermine the color of the patient's tooth shade.
 3. The method of claim2 wherein the patient's tooth shade is determined by selecting portionsof the electronically displayed image of the tooth; determining thecolor of the selected portions; and comparing that color informationwith the stored tooth shade color information to determine the color ofthat portion of the patient's tooth.
 4. The method of claim 3 whereinthe selection of portions of the tooth is repeated until a tooth shadecolor is determined for all spatial locations of the image of thepatient's tooth.
 5. The method of claim 4 wherein the patient's toothshade is determined by averaging the color information at selectedportions of the image before comparing the averaged color informationwith the stored tooth shade color information.
 6. The method of claim 1which further comprises utilizing a camera to obtain the image of thepatient's tooth.
 7. The method of claim 6 which further comprisesproviding a reference color adjacent the tooth for intensity correctionof the color information.
 8. The method of claim 6 wherein the colorinformation representative of the tooth shades is obtained from adigital camera and is stored electronically before the patient's toothis imaged.
 9. The method of claim 1 which further compriseselectronically forwarding the patient's tooth shade color(s) to a dentallaboratory for use in reconstruction of a tooth for the patient.
 10. Themethod of claim 9 wherein the laboratory manufactures the reconstructedtooth utilizing a plurality of porcelain coatings.